new: Support Bazel Remote APIs. #1651

milesj · 2024-09-13T23:56:07Z

Will start with caching.

harlequin · 2024-10-11T09:09:08Z

Hey,

I have seen that this feature was not introduced into the 1.29 release.
What are the missing pieces, despite the merge conflicts.

Maybe I can support here.

milesj · 2024-10-11T16:19:25Z

@harlequin The requests from moon to bazel-remote fail with cryptic http2 errors that I've been unable to figure out.

hyperium/tonic#1949

harlequin · 2024-10-12T15:53:59Z

As far as I remember HTTP2 implies that you have SSL enabled. Will try to setup an environment for testing.

milesj · 2024-10-13T04:04:16Z

Yeah I think the problem has to do with the bazel/tonic libraries being http2, while bazel-remote is http1, and the communication between the 2 just isn't working. This area isn't an expertise of mine.

harlequin · 2024-10-15T07:43:54Z

I have played around with the remote service ...

What I have found so far

1. TLS has to be activated and the endpoint has to include .assume_http2(true)

https://docs.rs/tonic/latest/tonic/transport/channel/struct.ClientTlsConfig.html#method.assume_http2

endpoint = endpoint
            .tls_config(
                ClientTlsConfig::new()
                    .with_enabled_roots()
                    .assume_http2(true)                    
            )
            .unwrap();

With this setting I can connect to the remote service

2. GZIP, ZSTD compression

So with the enabled connection moon tries to upload the cache into the remote service

bazel-remote-service shows

2024/10/15 07:37:48 POST 400 192.168.44.1 /build.bazel.remote.execution.v2.ContentAddressableStorage/BatchUpdateBlobs

But on moon side I am receiving

[crates\remote\src\cache_api.rs:96:13] &error = Status {
    code: Internal,
    message: "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request",
    metadata: MetadataMap {
        headers: {
            "content-type": "text/plain; charset=utf-8",
            "x-content-type-options": "nosniff",
            "date": "Tue, 15 Oct 2024 07:37:48 GMT",
            "content-length": "126",
            "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None",
        },
    },
    source: None,
}
[crates\remote\src\cache_api.rs:97:13] error.code() = Internal
[crates\remote\src\cache_api.rs:98:13] error.metadata() = MetadataMap {
    headers: {
        "content-type": "text/plain; charset=utf-8",
        "x-content-type-options": "nosniff",
        "date": "Tue, 15 Oct 2024 07:37:48 GMT",
        "content-length": "126",
        "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None",
    },
}
[crates\remote\src\cache_api.rs:99:13] error.message() = "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request"
Error:   × Main thread panicked.
  ├─▶ at C:\Users\rustmec\code\contributions\moon\crates\remote\src\cache_api.rs:101:13
  ╰─▶ Status { code: Internal, message: "protocol error: received message with invalid compression flag: 114 (valid flags are 0 and 1) while receiving response with status: 400 Bad Request", metadata: MetadataMap { headers: {"content-type": "text/plain; charset=utf-8", "x-content-type-    
      options": "nosniff", "date": "Tue, 15 Oct 2024 07:37:48 GMT", "content-length": "126", "set-cookie": "4afd39e673bbfe62f6f7f4a9622d53cd=9c659af26f02f4f78a396f6570abab67; path=/; HttpOnly; Secure; SameSite=None"} }, source: None }
  help: set the `RUST_BACKTRACE=1` environment variable to display a backtrace.

So what I have seen bazel-remote supports uncompressed and zstd for communication.
In moon is only gzip available as config.

My best guess by today, that something has to change additional to the compression ...

Hope this helps

harlequin · 2024-11-12T15:10:40Z

Hey @milesj

I have looked into the complete topic of bazel-remote-api and the specifications.

So this is what I have found out:

bazel defines 2 major protocol specifications, http1.1 (legacy) and grpc (new)

http1.1 is only handling CAS and AC storage via PUT, GET and HEAD commands reference

Implementing this would decrease the complexity inside moon and would have the benefit that mostly every http server can handle this kind of requests.

Even some people have implemented this directly with s3 storages. (can't find a good reference for it)

grpc is a next iteration for the bazel-remote-api, which includes many service like CAS, AC, remote-execution, log stream and so on.
the server which you are choosen bazel-remote is not the interface implemented for the asset api, here we shall choose another product
bazel differentiates the protocols with the https or grpcs protocol settings inside the url
the crate bazel-remote-api, which is used now in moon can only handle grpc protocol

Summary

I would go the way to split this tasks into, at least, 2 seperate Issues / Features

Implement bazel-remote-api with HTTP1.1 and reqwest crate (incl. reqwest-netrc) for authentication

This would have the benefit that the functionallity from moonbase in regards of remote-caching is equal and can be replaced.
Implement CAS and AC bazel-remote-api with GRPC

With this task you can define on the roadmap, which services moon will support, as mentioned CAS and AC would be a great starting point.

All other services from the bazel specification can introduced over time in the moon application.

milesj · 2024-11-12T20:23:41Z

Awesome work. I wasn't even aware that the bazel APIs had 2 separate protocols. But that definitely makes sense for the issues I've been having.

I also agree with splitting this up into 2 separate features. I'm going to try and focus on this for 1.30. It would be nice to land this and python support in the next release.

milesj · 2024-11-16T04:43:31Z

I got this working with grpc. An update here: #1520 (comment)

github-actions · 2024-11-16T04:54:11Z

Run report for b606888f `(ubuntu-latest, 22)`

Total time: 3m 43s | Comparison time: 5m 27s | Estimated savings: 1m 43s (31.6% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	40.9ms	Passed
🟩	`SetupToolchain(node:~22)`	47s	Passed
🟩	`SyncProject(node, types)`	3.8ms	Passed
🟩	`SyncProject(node, runtime)`	3ms	Passed
🟩	`SyncProject(node, visualizer)`	13.5ms	Passed
🟩	`SyncProject(node, report)`	5.7ms	Passed
🟩	`SyncProject(node, nx-compat)`	3.1ms	Passed
🟩	`SyncProject(node, website)`	3.5ms	Passed
🟩	`InstallWorkspaceDeps(node:~22)`	41.3s	Passed
🟩	`RunTask(types:test)`	1.6s	Passed
🟩	`RunTask(types:format)`	1.9s	Passed
🟩	`RunTask(visualizer:typecheck)`	4.7s	Passed
🟩	`RunTask(visualizer:test)`	1.6s	Passed
🟩	`RunTask(types:build)`	7.3s	Passed
🟩	`RunTask(types:typecheck)`	4.6s	Passed
🟩	`RunTask(types:lint)`	11.6s	Passed
🟩	`RunTask(visualizer:format)`	1.6s	Passed
🟩	`RunTask(nx-compat:format)`	1.4s	Passed
🟩	`RunTask(visualizer:lint)`	11.9s	Passed
🟩	`RunTask(report:typecheck)`	4.6s	Passed
	And 19 more...

Expanded report

	Action	Time	Status
🟩	`RunTask(visualizer:build)`	12.7s	Passed
🟩	`RunTask(nx-compat:build)`	7.6s	Passed
🟩	`RunTask(report:format)`	1.7s	Passed
🟩	`RunTask(runtime:test)`	1.5s	Passed
🟩	`RunTask(runtime:build)`	7s	Passed
🟩	`RunTask(runtime:format)`	1.1s	Passed
🟩	`RunTask(report:build)`	6.5s	Passed
🟩	`RunTask(runtime:typecheck)`	4.6s	Passed
🟩	`RunTask(report:test)`	3.9s	Passed
🟩	`RunTask(nx-compat:typecheck)`	5.3s	Passed
🟩	`RunTask(nx-compat:test)`	1.2s	Passed
🟩	`RunTask(runtime:lint)`	11.5s	Passed
🟩	`RunTask(report:lint)`	11.4s	Passed
🟩	`RunTask(nx-compat:lint)`	11.7s	Passed
🟩	`RunTask(website:typecheck)`	10.3s	Passed
🟩	`RunTask(website:test)`	1.5s	Passed
🟩	`RunTask(website:format)`	14.6s	Passed
🟩	`RunTask(website:lint)`	13.7s	Passed
🟩	`RunTask(website:build)`	1m 40s	Passed

Environment

OS: Linux
Matrix:

os = ubuntu-latest
node-version = 22

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

github-actions · 2024-11-19T03:06:39Z

Run report for b606888f `(ubuntu-latest, 18)`

Total time: 4m | Comparison time: 5m 47s | Estimated savings: 1m 46s (30.6% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	30.2ms	Passed
🟩	`SetupToolchain(node:~18)`	46.6s	Passed
🟩	`SyncProject(node, types)`	4.6ms	Passed
🟩	`SyncProject(node, runtime)`	9.8ms	Passed
🟩	`SyncProject(node, visualizer)`	16.9ms	Passed
🟩	`SyncProject(node, nx-compat)`	4.6ms	Passed
🟩	`SyncProject(node, report)`	13ms	Passed
🟩	`SyncProject(node, website)`	6.6ms	Passed
🟩	`InstallWorkspaceDeps(node:~18)`	42.6s	Passed
🟩	`RunTask(visualizer:typecheck)`	5.1s	Passed
🟩	`RunTask(types:test)`	1.9s	Passed
🟩	`RunTask(types:build)`	8s	Passed
🟩	`RunTask(visualizer:format)`	1.9s	Passed
🟩	`RunTask(types:typecheck)`	4s	Passed
🟩	`RunTask(types:lint)`	12.3s	Passed
🟩	`RunTask(types:format)`	1.8s	Passed
🟩	`RunTask(visualizer:build)`	13.3s	Passed
🟩	`RunTask(visualizer:test)`	1.3s	Passed
🟩	`RunTask(report:typecheck)`	4.9s	Passed
🟩	`RunTask(nx-compat:build)`	8.7s	Passed
	And 19 more...

Expanded report

	Action	Time	Status
🟩	`RunTask(visualizer:lint)`	12.9s	Passed
🟩	`RunTask(report:format)`	1.5s	Passed
🟩	`RunTask(report:lint)`	12.4s	Passed
🟩	`RunTask(runtime:build)`	7.3s	Passed
🟩	`RunTask(nx-compat:format)`	1.4s	Passed
🟩	`RunTask(runtime:format)`	1.1s	Passed
🟩	`RunTask(runtime:test)`	1.6s	Passed
🟩	`RunTask(report:build)`	7.7s	Passed
🟩	`RunTask(runtime:typecheck)`	5.5s	Passed
🟩	`RunTask(report:test)`	4.3s	Passed
🟩	`RunTask(nx-compat:typecheck)`	5.1s	Passed
🟩	`RunTask(nx-compat:test)`	1.6s	Passed
🟩	`RunTask(runtime:lint)`	11.3s	Passed
🟩	`RunTask(nx-compat:lint)`	11.8s	Passed
🟩	`RunTask(website:typecheck)`	11s	Passed
🟩	`RunTask(website:test)`	1.5s	Passed
🟩	`RunTask(website:format)`	18.3s	Passed
🟩	`RunTask(website:lint)`	13.6s	Passed
🟩	`RunTask(website:build)`	1m 50s	Passed

Environment

OS: Linux
Matrix:

os = ubuntu-latest
node-version = 18

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

github-actions · 2024-11-21T01:15:58Z

Run report for b606888f `(ubuntu-latest, 20)`

Total time: 3m 50s | Comparison time: 5m 41s | Estimated savings: 1m 50s (32.4% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	39.9ms	Passed
🟩	`SetupToolchain(node:~20)`	44.6s	Passed
🟩	`SyncProject(node, visualizer)`	5.6ms	Passed
🟩	`SyncProject(node, types)`	6.6ms	Passed
🟩	`SyncProject(node, runtime)`	2.8ms	Passed
🟩	`SyncProject(node, nx-compat)`	3.2ms	Passed
🟩	`SyncProject(node, website)`	3.1ms	Passed
🟩	`SyncProject(node, report)`	3ms	Passed
🟩	`InstallWorkspaceDeps(node:~20)`	43.4s	Passed
🟩	`RunTask(visualizer:format)`	1.7s	Passed
🟩	`RunTask(types:format)`	1.9s	Passed
🟩	`RunTask(types:test)`	1.4s	Passed
🟩	`RunTask(visualizer:typecheck)`	5s	Passed
🟩	`RunTask(types:build)`	7.5s	Passed
🟩	`RunTask(types:typecheck)`	4.8s	Passed
🟩	`RunTask(visualizer:test)`	1.4s	Passed
🟩	`RunTask(visualizer:build)`	13s	Passed
🟩	`RunTask(types:lint)`	12.3s	Passed
🟩	`RunTask(report:typecheck)`	4.9s	Passed
🟩	`RunTask(visualizer:lint)`	12s	Passed
	And 19 more...

Expanded report

	Action	Time	Status
🟩	`RunTask(nx-compat:build)`	8s	Passed
🟩	`RunTask(runtime:build)`	8.3s	Passed
🟩	`RunTask(report:format)`	1.9s	Passed
🟩	`RunTask(nx-compat:format)`	1.5s	Passed
🟩	`RunTask(report:build)`	7.5s	Passed
🟩	`RunTask(runtime:format)`	1.2s	Passed
🟩	`RunTask(report:lint)`	11.7s	Passed
🟩	`RunTask(runtime:test)`	1.5s	Passed
🟩	`RunTask(runtime:typecheck)`	4.6s	Passed
🟩	`RunTask(report:test)`	3.7s	Passed
🟩	`RunTask(nx-compat:test)`	1.4s	Passed
🟩	`RunTask(nx-compat:typecheck)`	5.4s	Passed
🟩	`RunTask(runtime:lint)`	11.6s	Passed
🟩	`RunTask(nx-compat:lint)`	12.3s	Passed
🟩	`RunTask(website:typecheck)`	11.1s	Passed
🟩	`RunTask(website:test)`	1.4s	Passed
🟩	`RunTask(website:format)`	15.3s	Passed
🟩	`RunTask(website:lint)`	14s	Passed
🟩	`RunTask(website:build)`	1m 50s	Passed

Environment

OS: Linux
Matrix:

os = ubuntu-latest
node-version = 20

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

github-actions · 2024-11-21T22:58:41Z

Run report for b606888f `(windows-latest, 18)`

Total time: 2m 15s | Comparison time: 5m 32s | Estimated savings: 3m 16s (59.1% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	38.3ms	Passed
🟩	`SetupToolchain(node:~18)`	55.9s	Passed
🟩	`SyncProject(node, types)`	5.3ms	Passed
🟩	`SyncProject(node, visualizer)`	5.8ms	Passed
🟩	`SyncProject(node, runtime)`	6ms	Passed
🟩	`SyncProject(node, nx-compat)`	6.3ms	Passed
🟩	`SyncProject(node, report)`	7.1ms	Passed
🟩	`SyncProject(node, website)`	4.3ms	Passed
🟩	`InstallWorkspaceDeps(node:~18)`	1m 3s	Passed
🟦	`RunTask(types:build)`	660ms	Cached
🟦	`RunTask(types:lint)`	775.8ms	Cached
🟦	`RunTask(visualizer:typecheck)`	777.3ms	Cached
🟦	`RunTask(visualizer:build)`	929.8ms	Cached
🟦	`RunTask(types:test)`	407.8ms	Cached
🟦	`RunTask(types:typecheck)`	406.3ms	Cached
🟦	`RunTask(visualizer:lint)`	377.3ms	Cached
🟦	`RunTask(visualizer:test)`	539.8ms	Cached
🟦	`RunTask(report:typecheck)`	524.3ms	Cached
🟦	`RunTask(nx-compat:build)`	536.4ms	Cached
🟦	`RunTask(runtime:build)`	547.2ms	Cached
	And 19 more...

Expanded report

	Action	Time	Status
🟩	`RunTask(visualizer:format)`	1.8s	Passed
🟦	`RunTask(report:build)`	445.6ms	Cached
🟩	`RunTask(types:format)`	2.3s	Passed
🟦	`RunTask(runtime:typecheck)`	531.2ms	Cached
🟩	`RunTask(runtime:format)`	1.9s	Passed
🟦	`RunTask(runtime:test)`	555.9ms	Cached
🟩	`RunTask(report:format)`	2.2s	Passed
🟩	`RunTask(nx-compat:format)`	2.1s	Passed
🟦	`RunTask(report:test)`	437.9ms	Cached
🟦	`RunTask(nx-compat:typecheck)`	549.8ms	Cached
🟦	`RunTask(runtime:lint)`	475.3ms	Cached
🟦	`RunTask(nx-compat:lint)`	389.7ms	Cached
🟦	`RunTask(nx-compat:test)`	459.6ms	Cached
🟦	`RunTask(report:lint)`	489.4ms	Cached
🟦	`RunTask(website:build)`	4.1s	Cached
🟦	`RunTask(website:lint)`	3.8s	Cached
🟦	`RunTask(website:test)`	3.7s	Cached
🟦	`RunTask(website:typecheck)`	456.4ms	Cached
🟩	`RunTask(website:format)`	10.9s	Passed

Environment

OS: Windows
Matrix:

os = windows-latest
node-version = 18

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

github-actions · 2024-11-21T22:59:21Z

Run report for b606888f `(windows-latest, 22)`

Total time: 2m 4s | Comparison time: 4m 35s | Estimated savings: 2m 31s (54.9% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	40.5ms	Passed
🟩	`SetupToolchain(node:~22)`	49.4s	Passed
🟩	`SyncProject(node, types)`	7.1ms	Passed
🟩	`SyncProject(node, visualizer)`	7.8ms	Passed
🟩	`SyncProject(node, report)`	6ms	Passed
🟩	`SyncProject(node, nx-compat)`	6ms	Passed
🟩	`SyncProject(node, runtime)`	7.4ms	Passed
🟩	`SyncProject(node, website)`	4.5ms	Passed
🟩	`InstallWorkspaceDeps(node:~22)`	59.2s	Passed
🟦	`RunTask(types:build)`	736.6ms	Cached
🟦	`RunTask(visualizer:typecheck)`	857.1ms	Cached
🟦	`RunTask(types:lint)`	864.9ms	Cached
🟦	`RunTask(visualizer:build)`	1s	Cached
🟦	`RunTask(types:test)`	427.3ms	Cached
🟦	`RunTask(types:typecheck)`	443.2ms	Cached
🟦	`RunTask(visualizer:lint)`	404.3ms	Cached
🟦	`RunTask(visualizer:test)`	420.2ms	Cached
🟦	`RunTask(report:typecheck)`	473.6ms	Cached
🟦	`RunTask(nx-compat:build)`	481.2ms	Cached
🟦	`RunTask(runtime:build)`	502.4ms	Cached
	And 19 more...

Expanded report

	Action	Time	Status
🟦	`RunTask(report:build)`	419.9ms	Cached
🟩	`RunTask(types:format)`	2.2s	Passed
🟩	`RunTask(visualizer:format)`	2.6s	Passed
🟦	`RunTask(runtime:test)`	440ms	Cached
🟦	`RunTask(runtime:typecheck)`	461.1ms	Cached
🟦	`RunTask(report:test)`	453.9ms	Cached
🟩	`RunTask(report:format)`	2.3s	Passed
🟩	`RunTask(nx-compat:format)`	2.1s	Passed
🟦	`RunTask(nx-compat:typecheck)`	657.5ms	Cached
🟦	`RunTask(runtime:lint)`	417.7ms	Cached
🟦	`RunTask(nx-compat:lint)`	572.2ms	Cached
🟦	`RunTask(nx-compat:test)`	428.7ms	Cached
🟦	`RunTask(report:lint)`	422.7ms	Cached
🟩	`RunTask(runtime:format)`	1.8s	Passed
🟦	`RunTask(website:lint)`	476.4ms	Cached
🟦	`RunTask(website:typecheck)`	966.2ms	Cached
🟦	`RunTask(website:test)`	652.4ms	Cached
🟦	`RunTask(website:build)`	4.5s	Cached
🟩	`RunTask(website:format)`	10.2s	Passed

Environment

OS: Windows
Matrix:

os = windows-latest
node-version = 22

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

github-actions · 2024-11-21T23:02:16Z

Run report for b606888f `(windows-latest, 20)`

Total time: 2m 7s | Comparison time: 5m 33s | Estimated savings: 3m 25s (61.7% faster)

	Action	Time	Status
🟩	`SyncWorkspace`	33.7ms	Passed
🟩	`SetupToolchain(node:~20)`	52s	Passed
🟩	`SyncProject(node, visualizer)`	5.8ms	Passed
🟩	`SyncProject(node, types)`	6.2ms	Passed
🟩	`SyncProject(node, report)`	5.6ms	Passed
🟩	`SyncProject(node, nx-compat)`	6.5ms	Passed
🟩	`SyncProject(node, runtime)`	4.9ms	Passed
🟩	`SyncProject(node, website)`	3.4ms	Passed
🟩	`InstallWorkspaceDeps(node:~20)`	1m	Passed
🟦	`RunTask(visualizer:build)`	705.6ms	Cached
🟦	`RunTask(types:lint)`	820.5ms	Cached
🟦	`RunTask(types:build)`	854ms	Cached
🟦	`RunTask(visualizer:typecheck)`	403.3ms	Cached
🟦	`RunTask(types:test)`	402.2ms	Cached
🟦	`RunTask(visualizer:lint)`	388.9ms	Cached
🟦	`RunTask(visualizer:test)`	405ms	Cached
🟩	`RunTask(visualizer:format)`	1.9s	Passed
🟦	`RunTask(types:typecheck)`	409.2ms	Cached
🟦	`RunTask(runtime:build)`	420.9ms	Cached
🟦	`RunTask(report:typecheck)`	415.5ms	Cached
	And 19 more...

Expanded report

	Action	Time	Status
🟦	`RunTask(nx-compat:build)`	546.9ms	Cached
🟦	`RunTask(report:build)`	418.8ms	Cached
🟩	`RunTask(types:format)`	2.1s	Passed
🟦	`RunTask(report:lint)`	433ms	Cached
🟦	`RunTask(runtime:typecheck)`	396.8ms	Cached
🟦	`RunTask(runtime:test)`	383.8ms	Cached
🟦	`RunTask(report:test)`	447.7ms	Cached
🟩	`RunTask(report:format)`	2.3s	Passed
🟦	`RunTask(nx-compat:typecheck)`	421.3ms	Cached
🟩	`RunTask(nx-compat:format)`	2.2s	Passed
🟦	`RunTask(runtime:lint)`	413.3ms	Cached
🟩	`RunTask(runtime:format)`	1.8s	Passed
🟦	`RunTask(nx-compat:lint)`	393.8ms	Cached
🟦	`RunTask(nx-compat:test)`	476.7ms	Cached
🟦	`RunTask(website:typecheck)`	494ms	Cached
🟦	`RunTask(website:build)`	4.7s	Cached
🟦	`RunTask(website:lint)`	4.5s	Cached
🟦	`RunTask(website:test)`	4.7s	Cached
🟩	`RunTask(website:format)`	10s	Passed

Environment

OS: Windows
Matrix:

os = windows-latest
node-version = 20

Touched files

.gitignore
.moon/workspace.yml
CHANGELOG.md
Cargo.lock
Cargo.toml
crates/action-pipeline/Cargo.toml
crates/action-pipeline/src/action_pipeline.rs
crates/action-pipeline/src/subscribers/mod.rs
crates/action-pipeline/src/subscribers/remote_subscriber.rs
crates/action/src/operation.rs
crates/actions/Cargo.toml
crates/actions/src/actions/install_deps.rs
crates/actions/src/actions/sync_workspace.rs
crates/app/Cargo.toml
crates/app/src/session.rs
crates/cache/src/cache_engine.rs
crates/cache/src/hash_engine.rs
crates/cache/tests/hash_engine_test.rs
crates/cli/src/main.rs
crates/config/src/shapes/poly.rs
crates/config/src/workspace/mod.rs
crates/config/src/workspace/remote_config.rs
crates/config/src/workspace_config.rs
crates/hash/Cargo.toml
crates/remote/Cargo.toml
crates/remote/src/fs_digest.rs
crates/remote/src/grpc_remote_client.rs
crates/remote/src/grpc_tls.rs
crates/remote/src/lib.rs
crates/remote/src/remote_client.rs
crates/remote/src/remote_error.rs
crates/remote/src/remote_service.rs
crates/remote/tests/__fixtures__/certs-local/README.md
crates/remote/tests/__fixtures__/certs-local/ca.crt
crates/remote/tests/__fixtures__/certs-local/ca.key
crates/remote/tests/__fixtures__/certs-local/ca.pem
crates/remote/tests/__fixtures__/certs-local/client.crt
crates/remote/tests/__fixtures__/certs-local/client.csr
crates/remote/tests/__fixtures__/certs-local/client.key
crates/remote/tests/__fixtures__/certs-local/client.pem
crates/remote/tests/__fixtures__/certs-local/server.crt
crates/remote/tests/__fixtures__/certs-local/server.csr
crates/remote/tests/__fixtures__/certs-local/server.key
crates/remote/tests/__fixtures__/certs-local/server.pem
crates/remote/tests/__fixtures__/certs/README.md
crates/remote/tests/__fixtures__/certs/ca.pem
crates/remote/tests/__fixtures__/certs/client.key
crates/remote/tests/__fixtures__/certs/client.pem
crates/remote/tests/__fixtures__/certs/server.key
crates/remote/tests/__fixtures__/certs/server.pem
crates/task-runner/Cargo.toml
crates/task-runner/src/output_archiver.rs
crates/task-runner/src/output_hydrater.rs
crates/task-runner/src/run_state.rs
crates/task-runner/src/task_runner.rs
crates/task-runner/tests/output_archiver_test.rs
crates/task-runner/tests/output_hydrater_test.rs
crates/task-runner/tests/task_runner_test.rs
crates/task-runner/tests/utils.rs
crates/task/src/task.rs
crates/workspace/src/workspace_builder.rs
justfile
packages/types/src/project-config.ts
packages/types/src/workspace-config.ts
scripts/data/generateCerts.sh
tests/docker/Dockerfile
tests/docker/Dockerfile.staged
website/static/schemas/workspace.json

* Add crate. * Add config. * Start on cache. * Move channel. * Hook up service. * Start on new clients. * Get things working. * Update configs. * Start on action results. * Calculate outputs. * Get archiving working. * Upload in background. * Start on downloading. * Polish error handling. * Instrument. * Fix lints. * Update tests. * Start on tls/mtls. * Group uploads. * Group downloads. * Smart partition groups. * Fix cert roots. * Polish.

milesj force-pushed the develop-1.29 branch 2 times, most recently from 4d62cd9 to f44f5d0 Compare September 24, 2024 01:11

Base automatically changed from develop-1.29 to master October 7, 2024 16:09

milesj added 5 commits November 15, 2024 10:24

Add crate.

ddf781d

Add config.

c6a8cb0

Start on cache.

9312ba7

Move channel.

298151d

Hook up service.

df309a3

milesj force-pushed the 1.29-bazel branch from 10c4a66 to df309a3 Compare November 15, 2024 18:29

milesj changed the base branch from master to develop-1.30 November 15, 2024 18:29

Start on new clients.

979ee55

milesj mentioned this pull request Nov 16, 2024

[feature] Support Remote Execution API for caching #1520

Closed

milesj added 2 commits November 15, 2024 20:45

Get things working.

4de3261

Update configs.

efd3027

milesj added 3 commits November 16, 2024 17:04

Start on action results.

470ec3c

Calculate outputs.

47f19b3

Get archiving working.

1d93002

milesj added 3 commits November 19, 2024 20:01

Upload in background.

19f2950

Start on downloading.

6ddb66a

Polish error handling.

7544e83

milesj added 2 commits November 20, 2024 16:57

Instrument.

7086a96

Fix lints.

e0bc990

Update tests.

9f5dbfe

milesj added 6 commits November 22, 2024 16:24

Start on tls/mtls.

73419c4

Group uploads.

6ae1263

Group downloads.

cd0c9f7

Smart partition groups.

54b16c2

Fix cert roots.

c5ced29

Polish.

b606888

milesj merged commit 7885e7c into develop-1.30 Nov 23, 2024
27 checks passed

milesj deleted the 1.29-bazel branch November 23, 2024 15:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

new: Support Bazel Remote APIs. #1651

new: Support Bazel Remote APIs. #1651

milesj commented Sep 13, 2024

harlequin commented Oct 11, 2024

milesj commented Oct 11, 2024 •

edited

Loading

harlequin commented Oct 12, 2024

milesj commented Oct 13, 2024 •

edited

Loading

harlequin commented Oct 15, 2024

harlequin commented Nov 12, 2024

milesj commented Nov 12, 2024

milesj commented Nov 16, 2024

github-actions bot commented Nov 16, 2024 •

edited

Loading

github-actions bot commented Nov 19, 2024 •

edited

Loading

github-actions bot commented Nov 21, 2024 •

edited

Loading

github-actions bot commented Nov 21, 2024 •

edited

Loading

github-actions bot commented Nov 21, 2024 •

edited

Loading

github-actions bot commented Nov 21, 2024 •

edited

Loading

new: Support Bazel Remote APIs. #1651

new: Support Bazel Remote APIs. #1651

Conversation

milesj commented Sep 13, 2024

harlequin commented Oct 11, 2024

milesj commented Oct 11, 2024 • edited Loading

harlequin commented Oct 12, 2024

milesj commented Oct 13, 2024 • edited Loading

harlequin commented Oct 15, 2024

harlequin commented Nov 12, 2024

Summary

milesj commented Nov 12, 2024

milesj commented Nov 16, 2024

github-actions bot commented Nov 16, 2024 • edited Loading

Run report for b606888f (ubuntu-latest, 22)

github-actions bot commented Nov 19, 2024 • edited Loading

Run report for b606888f (ubuntu-latest, 18)

github-actions bot commented Nov 21, 2024 • edited Loading

Run report for b606888f (ubuntu-latest, 20)

github-actions bot commented Nov 21, 2024 • edited Loading

Run report for b606888f (windows-latest, 18)

github-actions bot commented Nov 21, 2024 • edited Loading

Run report for b606888f (windows-latest, 22)

github-actions bot commented Nov 21, 2024 • edited Loading

Run report for b606888f (windows-latest, 20)

milesj commented Oct 11, 2024 •

edited

Loading

milesj commented Oct 13, 2024 •

edited

Loading

github-actions bot commented Nov 16, 2024 •

edited

Loading

Run report for b606888f `(ubuntu-latest, 22)`

github-actions bot commented Nov 19, 2024 •

edited

Loading

Run report for b606888f `(ubuntu-latest, 18)`

github-actions bot commented Nov 21, 2024 •

edited

Loading

Run report for b606888f `(ubuntu-latest, 20)`

github-actions bot commented Nov 21, 2024 •

edited

Loading

Run report for b606888f `(windows-latest, 18)`

github-actions bot commented Nov 21, 2024 •

edited

Loading

Run report for b606888f `(windows-latest, 22)`

github-actions bot commented Nov 21, 2024 •

edited

Loading

Run report for b606888f `(windows-latest, 20)`